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Abstract 

This report focuses on the initial development of the predicted score paths for ACT Aspire 
reporting. The paths provide predicted score ranges for the next two years - as well as predicted 
ACT score ranges for tests administered at grades 9 and 10. Longitudinal ACT Aspire data for 
students tested in spring 2013 and spring 2014, as well as historical data from ACT’s college and 
career readiness assessment system, are used to develop the predicted paths. Coverage rates of 
the predicted paths are examined, and proposed uses and limitations of the predicted paths are 
discussed. 
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Introduction 

ACT Aspire will report student scores over multiple years, providing longitudinal progress 
reporting from grades 3-10. Score reports will include: 

• A student’s current and prior year scores in all tested subjects (English, Mathematics, 
Reading, Science, and Writing) 

• Comparison to ACT Readiness Benchmarks that indicate whether students are on target 
to meet ACT’s College Readiness Benchmarks in spring of grade 1 1 

• Predicted score paths that provide ranges for a student’s expected scores in future years 

• Predicted ACT score ranges (for grades 9 and 10) 

• Classification of student growth as “low”, “average”, or “high” based on student growth 
percentiles 

This report focuses on the development of the predicted score paths - which provide 
predicted score ranges for the next two years - as well as the predicted ACT score ranges. A 
prototype report that illustrates the predicted path is given in Figure 1. The predicted path is 
represented by a cone-shaped orange-shaded area that covers two years. In this example, an 
eighth grade student scored 417 on the ACT Aspire English Assessment. Her predicted path 
covers the score range 416-424 for ninth grade and 415-431 for tenth grade. 
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Figure 1 

ACT Aspire Prototype Student Progress Report 
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In addition to the student reports, predicted paths will also be provided for classroom, 
school, district, and state aggregate progress reports. For aggregate reports, students are grouped 
according to their current year classroom, school, or district. The predicted paths will only cover 
one year and will show the group’s predicted mean score (see Figure 2 for an excerpt from an 
example aggregate report). The aggregate predicted paths are represented by a line segment with 
the current year mean score as one endpoint (orange-filled circle) and the next year’s predicted 
mean score as the other endpoint (orange-outlined circle). For grades 9 and 10, ACT Aspire will 
also report predicted mean ACT scores, as shown in Figure 2. 
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Figure 2 

ACT Aspire Prototype District Progress Report 
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Predicted paths will be reported to enhance understanding of where a student (or a group of 
students) is likely to score in future years, assuming typical growth. This information can be 
used to: 

• Determine if students are likely to meet ACT Readiness Benchmarks over the next two 
years 

• Determine which students are unlikely to meet a future-year standard and thus are 
candidates for extra academic support 

• Predict aggregate future achievement for a classroom, school, district, or state 

• Predict ACT score ranges (for grades 9 and 10) 

In this report, the procedures used to develop the predicted paths are documented and some 
statistical properties of the predicted paths are examined. 

Methods 

Longitudinal Samples 


The ACT Aspire summative assessments were first administered in spring 2013 as part of 


special studies. In spring 2014, ACT Aspire summative assessments were launched. The sample 
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of students who tested in spring 2013 overlaps with those tested in spring 2014. Moreover, many 
of the 10 th grade students who tested in spring 2013 took the ACT Assessment in spring 2014. 
The students who tested in 2013 and 2014 form a longitudinal sample of ACT Aspire-tested 
students. The ACT Aspire-tested longitudinal sample sizes are given in Table 1 for each subject 
and grade level pair for which predicted paths will be developed. 

Table 1 

ACT Aspire-tested longitudinal sample sizes 


Grade Level 
Pair 

Subject Area 

English 

Mathematics 

Reading 

Science 

Writing 

Composite 

3-4 

3,784 

8,843 

8,943 

3,403 

469 


4-5 

3,912 

8,724 

8,694 

3,070 

487 


5-6 

3,563 

7,982 

8,008 

3,441 

431 


6-7 

2,329 

5,876 

5,880 

1,677 

381 


7-8 

1,924 

4,583 

4,579 

2,132 

262 


8-9 

241 

261 

118 

156 

116 

34 

9-10 

26 

21 

20 

21 

0 

15 

9-11 (ACT) 

0 

0 

0 

0 

0 

0 

10-11 (ACT) 

3,992 

3,774 

3,540 

3,525 

922 

2,851 


Ideally, the development of predicted score ranges would be based on large longitudinal 
samples of ACT Aspire-tested students. From Table 1, we see that the sample sizes vary 
considerably across grade levels and subject areas. In spring 2014, the state of Alabama 
administered ACT Aspire Mathematics and Reading to all students in grades 3-8. Many districts 
in Alabama also administered English, Science, and Writing to grade 3-8 students. Thus, the 
longitudinal sample sizes are larger for the pre-high school grade level pairs and considerably 
smaller for grade level pairs 8-9 and 9-10. Moreover, there are relatively few students who took 
ACT Aspire Writing tests in both 2013 and 2014. Because students have been assessed for at 
most two years as of spring 2014, there are no students who have taken ACT Aspire in grade 9 
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and the ACT in grade 11. For grades 3-7, ACT Aspire Composite scores will not be reported 
and so sample sizes are not given. 

ACT has developed a concordance table for estimating ACT Aspire scores from ACT 
Explore or ACT Plan scores (http://www.act.org/migrate/pdf/ConcordanceTable.pdf). In recent 
years, there are large samples of students who tested in grades 8-11 with ACT Explore, ACT 
Plan, and/or the ACT. After using the concordance tables to estimate ACT Aspire scores from 
ACT Explore or ACT Plan scores, these samples can be used to generate concordance-derived 
longitudinal samples of ACT Aspire scores. This approach was applied to large samples of 
students who completed high school in 2014. The one-year grade level pairs include students 
who tested 10-14 months apart; the 9-11 grade level pair includes students who tested 22-26 
months apart. The concordance-derived longitudinal sample sizes are given in Table 2 for each 
subject and grade level pair. 

Table 2 


Concordance-derived longitudinal samples of ACT Aspire scores 


Grade 

Level 

Pair 

Test 1 

Test 2 

Subject Area 

English 

Mathematics 

Reading 

Science 

Composite 

8-9 

ACT 

Explore 

ACT 

Explore 

53,537 

53,537 

53,537 

53,537 

53,537 

9-10 

ACT 

Explore 

ACT Plan 

172,339 

172,339 

172,339 

172,339 

172,339 

9-11 

ACT 

Explore 

ACT 

50,656 

50,656 

50,656 

50,656 

50,656 

10-11 

ACT Plan 

ACT 

148,356 

148,356 

148,356 

148,356 

148,356 


Note that Writing scores are not represented in the concordance-derived samples because 
neither ACT Explore nor ACT Plan includes a Writing test. Also note that the sample sizes are 
constant across subject areas because the Explore, Plan, and the ACT test batteries are 
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administered with all four subjects and students with missing test scores were deleted from the 
analysis file. 

Statistical Methods 

Given the available data and the intended uses of the predicted paths, we chose to base 
the one-year predicted paths on the estimated 25 th and 75 th percentile of the test score 
distribution, conditional on the prior year test score. Using an inclusive definition for predicted 
path coverage (scores greater than or equal to the lower score of the projected range and scores 
less than or equal to the upper score of the range), more than 50% of test scores would be 
expected to fall within the one- year predicted path. Less than 25% of test scores would be above 
the one-year predicted path score range and less than 25% would be below. 

The full predicted path, which encompasses the one-year and two-year predictions (see 
Figure 1), is drawn by extending the one- year predictions for another year in a linear fashion. 

Let C x represent the current year score at grade g, and let P 25 and P 75 represent the 25 th and 75 th 
percentiles of the distribution of grade g+1 scores, conditional on a grade g test score of C x . The 
predicted path is thus drawn as a triangle with vertices (g, C x ), (g+ 2, 2 P 25 - C x ,), (g+ 2, 2 P75- C x ). 

One-year predicted mean scores will be used to form aggregate predicted paths, and the 
aggregate predicted paths will be drawn as line segments as opposed to triangles (see Figure 2). 
Individual student predicted scores are based on the estimated 50 th percentile of the test score 
distribution (P 50 ), conditional on the prior year test score (C x ). Predicted mean scores for 
classrooms, schools, and districts are calculated as the mean of individual student predicted 
scores. Unlike the predicted paths for student score reports, the aggregate paths will not convey 
information about the uncertainty of the prediction. For classrooms, schools, and districts, two 
primary sources of uncertainty in predicted scores are 1) variation across units in expected 



growth, and 2) sampling error within units that is generally larger with smaller sample size. 
Future research may attempt to develop predicted paths for aggregates that convey uncertainty. 

To estimate P 25 , P 50 , and P 75 , quantile regression (Koenker & Bassett, 1978; Koenker, 
2005) was used. Quantile regression is conceptually similar to ordinary least-squares regression, 
which is used to estimate the mean of an outcome ( Y) given a set of predictor variables (denoted 
X). Quantile regression estimates selected quantiles of an outcome or dependent variable 
(denoted Y), given a set of predictor variables (denoted X). The quantile regression models were 
estimated using the QUANTREG procedure in SAS (Chen, 2005). 

For our purposes, quantile regression was used to estimate selected quantiles of spring 
2014 test scores (denoted F)> given specific values of spring 2013 test scores (denoted X). The 
desired quantiles (0.25, 0.50, and 0.75 corresponding to the 25 th , 50 th , and 75 th percentiles) are 
specified in the SAS QUANTREG procedure. 

For all subject areas except Writing, separate regression models were fit for each grade 
level pair and each subject area. For three of the grade level pairs (8-9, 9-10, 9-11), the 
concordance-derived longitudinal samples were used because the sample sizes for the ACT 
Aspire-tested longitudinal samples (Table 1) were too small. For the 10-11 grade level pair, the 
Aspire-ACT matched data sets were used for all subjects (including Writing) and the Composite 
score. 

For Writing, a single model was fit that combined data from all grade level pairs (3-4 
through 8-9, yielding a total sample size of 2,146). To allow for potential grade level effects, 
grade-level dummy variables were also included in the model, resulting in grade level pair- 
specific intercepts. No longitudinal Writing data exist for grade 9-10, so it is assumed that the 
quantile regression equations for grade level pairs 8-9 and 9-10 are equal. Because no data are 
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available to estimate predicted ACT Writing scores for grade 9 ACT Aspire Writing scores, the 
predicted ACT Writing scores for grade 10 Writing scores are also used for grade 9. 

Table 3 

Longitudinal samples used to develop predicted paths using quantile regression 


Grade 

Level 

Pair 

Subject Area 

English 

Mathematics 

Reading 

Science 

Writing 1 

Composite 

3-4 

3,784 

8,843 

8,943 

3,403 

469 


4-5 

3,912 

8,724 

8,694 

3,070 

487 


5-6 

3,563 

7,982 

8,008 

3,441 

431 


6-7 

2,329 

5,876 

5,880 

1,677 

381 


7-8 

1,924 

4,583 

4,579 

2,132 

262 


8-9 

*53,537 

*53,537 

*53,537 

*53,537 

116 

*53,537 

9-10 

*172,339 

*172,339 

*172,339 

*172,339 

0 

*172,339 

9-11 

(ACT) 

*50,656 

*50,656 

*50,656 

*50,656 

0 

*50,656 

10-11 

(ACT) 

3,992 

3,774 

3,540 

3,525 

922 

2,851 


'^Indicates concordance-derived sample 


Evaluating the Results 

Checks were done to examine the logical consistency of the predicted paths. The 
following propositions were examined: 

1) For scores that are at the ACT Readiness Benchmark, the predicted paths should include 
the ACT Readiness Benchmarks for the next two grade levels. Given the interpretation 
of the ACT Readiness Benchmarks as the scores needed to be on target for college and 
career readiness by spring grade 1 1, it is expected that students who meet a Benchmark 
should have a reasonable chance of also meeting the Benchmark the next two years. 

2) The one-year predicted path values (lower, upper, and predicted value) should be 
monotonically increasing across the score scale within each grade level and subject. 


1 For grade level pairs 3-4 through 8-9, data were combined for fitting the quantile regression models. 
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Moreover, for grades 9 and 10, predicted ACT scores (lower, upper, and predicted value) 
should be monotonically increasing across the score scale within each grade level and 
subject. 

3) For each grade 9 score, the predicted ACT score range should be at least as wide as the 
predicted ACT score ranges for the same grade 10 score. Because grade 10 is in closer 
proximity to the grade 1 1 ACT, there should be less variability in predicted ACT scores 
for grade 10 scores relative to grade 9 scores. 

4) For each grade 9 score, the predicted ACT score values should be greater than the 
predicted ACT score values for the same grade 10 score. Because grade 10 is in closer 
proximity to the grade 1 1 ACT, there is less time for academic growth to occur, and so 
the predicted ACT scores should be smaller for grade 10 scores relative to those for grade 
9 scores. 

If any of the consistency propositions were not met, but were nearly met, hand- 
adjustments would be made to the predicted paths to correct for the inconsistency. If any of the 
consistency propositions failed considerably, the methodology would need to be revisited. 

Following the consistency checks and any hand adjustments, descriptive statistics were 
examined including score means and standard deviations, correlations, gain score means and 
standard deviations, and proportion of examinees with positive gain scores. For all subject areas 
except Writing, ACT Aspire scores share a common scale across grade levels and positive mean 
gain scores are anticipated because students are expected to increase their knowledge and skills 
in the tested areas after one year of schooling. The ACT Aspire Writing tests have four domains, 
each of which is scored using a rubric with a score of “4” indicating expected grade level 
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performance. The domain scores determine a total raw score for the Writing test, which is then 
converted to a scale score. The scaling and scoring process for Writing implies that one should 
not necessarily expect positive gain scores, and that gain scores do not have a clear meaning for 
Writing. Rather, consistency in ACT Aspire Writing scores across grade levels suggests the 
same performance relative to grade level expectations. 

Following the descriptive statistics, coverage properties of the predicted paths were examined, 
including: 

1) The percentage of students scoring below, within, and above one-year ACT Aspire 
predicted paths. For this analysis, the ACT Aspire longitudinal sample was used (sample 
sizes given in Table 1) to evaluate the predicted paths. For grade level pairs 3-4 through 
7-8, and 10-11, the samples are used both for developing the predicted paths and for 
evaluating the predicted paths. Ideally, the development and evaluation samples would 
be independent. However, given that only one year of longitudinal data are available 
with limited sample sizes, this was not feasible. For grade level pairs 8-9, 9-10, and 9-11, 
the concordance-derived longitudinal sample was used to develop the predicted paths and 
the ACT Aspire longitudinal sample was used to evaluate the predicted paths. Thus, the 
development and evaluation samples are independent. However, for grade level pairs 8-9 
and 9-10, the evaluation samples are small. 

2) The percentage of students scoring below, within, and above one-year and two-year 
predicted paths derived using EPAS data (EPAS is the former name of ACT’s 
longitudinal assessment system that features ACT Explore, ACT Plan, and the ACT). 

This analysis did not use ACT Aspire data, but rather relied on recent EPAS data. The 
same procedure used to develop predicted paths for ACT Aspire was applied to the EPAS 
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data. Because multi-year longitudinal samples are available, this data allow us to study 
the properties of two-year predicted paths. While not directly applicable to ACT Aspire, 
the results should provide a general understanding of the properties of predicted paths 
that are developed using the stated procedures. Students who completed high school in 
2013 were used to develop the predicted paths and students who completed high school 
in 2014 were used to evaluate the predicted paths. This analysis was done for grade level 
pairs 8-9, 9-10, 10-11, 8-10, and 9-11. 

3) The percentage of students whose predicted path would predict strictly positive or strictly 
negative score gain. ACT Aspire features a vertical scale and positive gain scores (the 
arithmetic difference of the current year score and prior year score) suggest growth. 

While positive gain scores are more likely to occur and are consistent with the generally 
held idea that K-12 students gain knowledge and skills after one year of schooling, a 
combination of factors can lead to negative gain scores, including: 

a. Test measurement error. A student who scores above his true score in grade k and 
below his true score in grade k + 1 is more likely to see a decline in his test score. 

b. No change or a decline in the knowledge and skills tested during the time between 
tests. 

Therefore, it may be reasonable for predicted score paths to include negative gain scores. 
For this analysis, we used ACT Aspire score results from spring 2013 and assigned predicted 
paths to all scores. We then classified each prediction as A) strictly positive if the one-year 
predicted path only included positive gain scores, B) strictly negative if the one-year predicted 
path only included negative gain scores, and C) other if the predicted path included both positive 
and negative gain scores or if the predicted path included gain scores of 0. 



13 


Results 

Logical Consistency Checks 

Two of the four logical consistency checks were met with no exceptions: 1) For scores 
that are at the ACT Readiness Benchmark, all of the predicted paths included the ACT Readiness 
Benchmarks for the next two grade levels and 2) The one- year predicted path values (lower, 
upper, and predicted value) were all monotonically increasing across the score scale within each 
grade level and subject and, for grades 9 and 10, predicted ACT scores (lower, upper, and 
predicted value) were all monotonically increasing across the score scale within each grade level 
and subject. 

The other two consistency checks dealt with logical agreement between ACT score 
predictions for grade 9 and 10 scores. For each grade 9 score, the predicted ACT score range 
should be at least as wide as the predicted ACT score ranges for the same grade 10 score. And, 
for each grade 9 score, the predicted ACT score values should be greater than the predicted ACT 
score values for the same grade 10 score. These two propositions were not met in all cases. 
However, the deviations from the propositions were minor and so hand adjustments were applied 
to correct the predicted values. The ACT score predictions for grade 10 were developed using 
samples of students tested with ACT Aspire and the ACT, whereas the score predictions for 
grade 9 were developed using the concordance-derived samples. Thus, the grade 10 predictions 
were deemed more reliable and the hand adjustments were made to the grade 9 values. The hand 
adjustment process involved widening some of the grade 9 predicted ACT score ranges (usually 
by one score point) and increasing some of the grade 9 predicted ACT score values. The vast 
majority of hand adjustments to the predicted ACT score ranges were made at the extremes of 
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the score distributions (very low scores and very high scores). All of the hand adjustments to the 
predicted ACT score values were made in the lower extremes of the score distributions. 

Descriptive Statistics 

The gender and racial/ethnic breakdowns of the longitudinal samples are given in Table 
4. Male and female students are evenly represented, with the largest imbalance observed for 
grade 10-11 (51.8% female, 46.7% male). The early grade level pairs (3-4 through 7-8) are 
predominantly White (57.5%-62.1%) or Black/ African American (25.8% to 32.0%). The 
percentage of students of Hispanic ethnicity ranged from 4.7% to 6.0% for grades 3-4 through 7- 
8, but was much higher in the concordance-derived samples (15.3% to 27.2%). The share of 
Black/ African American students was smaller in the concordance-derived samples (8.8% to 
16.6%), but was larger in the grade 10-11 sample (25.6%). Across grade levels, there were small 
percentages of students from the other racial/ethnic groups (American Indian/ Alaska Native, 
Asian, Native Hawaiian/Pacific Islander, and two or more races). 

Table 4 

Gender and race/ethnicity of students in longitudinal samples 


Group 

Grade Level Pair 

3-4 

4-5 

5-6 

6-7 

7-8 

8-9 

9-10 

9-11 

10-11 

Gender 










Female 

48.7 

49.1 

49.9 

48.1 

50.0 

50.2 

49.7 

50.6 

51.8 

Male 

51.2 

50.7 

49.7 

51.8 

49.8 

49.8 

50.2 

49.4 

46.7 

Missing 

0.1 

0.2 

0.4 

0.0 

0.2 

0.0 

0.0 

0.0 

1.5 

Race/ethnicity 










American Indian/ Alaska Native 

1.5 

1.5 

1.9 

1.8 

1.5 

0.5 

0.5 

0.4 

0.2 

Asian 

2.1 

1.9 

1.4 

1.2 

1.3 

4.0 

4.0 

3.4 

2.7 

Black/ African American 

27.1 

25.8 

27.9 

28.6 

32.0 

16.6 

12.3 

8.8 

25.6 

Hispanic/Latino 

6.0 

5.2 

5.1 

4.8 

4.7 

27.2 

20.2 

15.3 

7.9 

Native Hawaiian/Pacific 
Islander 

0.1 

0.1 

0.3 

0.1 

0.0 

0.2 

0.3 

0.3 

0.2 

Two or more races 

0.0 

0.0 

0.0 

0.0 

0.0 

3.8 

4.2 

4.5 

4.1 

White 

59.7 

62.1 

59.3 

61.0 

57.5 

47.2 

57.1 

66.5 

53.2 

Missing 

3.6 

3.4 

4.1 

2.5 

3.0 

0.6 

1.5 

0.8 

6.1 
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The longitudinal samples for the early grade levels (3-4 through 7-8) come almost 
exclusively from the East region of the United States (Table 5), owing to Alabama’s adoption of 
ACT Aspire for grades 3-8 in spring 2014. The concordance-derived samples (grades 8-9, 9-10, 
and 9-11) come mostly from the Midwest (67.1% to 82.5%) and West (13.7% to 22.3%). The 
grade 10 to 11 (Aspire to ACT) sample comes mostly from the East (55.8%) and Midwest 
(34.59%). 

Table 5 


Regions represented in longitudinal samples 


Region 

Grade Level Pair 

3-4 

4-5 

5-6 

6-7 

7-8 

8-9 

9-10 

9-11 

10-11 

East 

99.7 

99.1 

98.2 

99.5 

99.0 

1.9 

5.9 

2.4 

55.8 

Midwest 

0.3 

0.4 

0.5 

0.5 

1.0 

76.3 

67.1 

82.5 

34.5 

Southwest 



0.1 

<0.1 


3.7 

4.8 

1.4 

6.3 

West 


0.5 

1.2 



18.1 

22.3 

13.7 

3.5 


Descriptive statistics of the test scores are provided in Tables 6-9. The prior score means 
and standard deviations (Table 6) are used to assess the achievement distribution of the samples 
used to develop the predicted paths. Estimates of national mean scores , which will be included 
on ACT Aspire score reports, are presented below the sample mean and standard deviation. In 
most cases, the sample mean matches the national mean within one score point. The grade 8 
mean scores for the grade 8-9 sample are smaller than the national means for English (424.3 vs. 
427), Reading (418.3 vs. 422), and Science (420.8 vs. 423), but are higher for Writing (426.7 vs. 


2 Note that the region definitions used by ACT and used in this report are different than those used by the U.S. 
Census Bureau, with large shares of states assigned to the East region (20 states plus the District of Columbia) and 
West region (16 states). 

3 The national mean scores were estimated using ACT Aspire test scores from 2013 and 2014 for students who took 
the on grade test (e.g., 5 th grade students who took the 6 th grade test were excluded). The sample was not reweighted 
to match national demographics. 
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423). The grade 10 mean scores for the grade 10-11 sample are smaller than the national means 
for all subject areas, with Reading and Science having the largest differences. 

Table 6 

Prior Score Means and Standard Deviations, National Means 


Grade 

Level 

Pair 

Subject Area 

English 

Mathematics 

Reading 

Science 

Writing 

Composite 

3-4 

416.6(5.8) 

416 

412.2 (3.9) 
412 

412.3 (5.2) 
412 

414.1 (6.0) 
414 

423.5 (5.7) 
421 


4-5 

420.0 (6.0) 
420 

414.9 (3.9) 
415 

414.6 (5.6) 
415 

416.8 (6.6) 
417 

424.3 (6.6) 
423 


5-6 

421.7 (6.8) 
422 

416.8 (4.7) 
417 

416.5 (6.2) 
417 

418.5 (6.5) 
419 

421.3 (6.9) 
424 


6-7 

423.3 (7.5) 
424 

418.1 (5.5) 
419 

418.5 (6.6) 
419 

418.4 (7.2) 
420 

427.1 (7.1) 
426 


7-8 

424.3 (8.6) 
425 

419.2 (6.6) 
419 

418.2 (6.7) 
419 

419.1 (7.6) 
420 

427.4 (6.5) 
423 


8-9 

424.3 (9.3) 
427 

421.7 (7.8) 
421 

418.3 (6.7) 
422 

420.8 (7.1) 
423 

426.7 (6.2) 
423 

421.4 (6.9) 

9-10 

428.0 (9.6) 
428 

424.2 (7.8) 
424 

420.9 (7.2) 
421 

423.7 (8.0) 
424 


424.3 (7.3) 

9-11 (ACT) 

429.5 (9.2) 
428 

425.5 (7.7) 
424 

422.1 (7.0) 
421 

425.3 (7.8) 
424 


425.7 (7.1) 

10-11 

(ACT) 

429.8 (11.0) 
431 

425.5 (9.1) 
427 

421.1 (8.0) 
423 

424.1 (8.8) 
426 

426.3 (7.9) 
427 

425.6 (8.2) 


Longitudinal test score correlations are provided in Table 7. For English and 
Mathematics, correlations for grades 3-4 through 7-8 are lower than those observed for grades 8- 
9, 9-10, and 9-11. Because the correlations for grades 8-9, 9-10, and 9-11 are based on the 
concordance-derived samples, we do not know if the correlation difference is due to a true 
difference in the longitudinal correlations across grade levels, or an artifact of using a 
concordance-derived sample. For English and Mathematics, the correlations generally increase 
with grade level. For Reading and Science, the correlations are very steady across grade levels, 


typically between 0.70 and 0.75. Across subject areas, the smallest correlations are observed for 
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Writing, ranging from 0.32 for grades 5-6 to 0.64 for grades 8-9. Given that the Writing tests 
require an essay response to a single writing prompt, the lower correlations are not surprising. 
Other possible reasons for the lower correlations for Writing include the changes across grade 
levels in the writing task and the use of rubric scoring. The Writing correlations are also subject 
to greater sampling error due to the smaller sample sizes. Correlations between grade 10 ACT 
Aspire scores and grade 1 1 ACT scores ranged from 0.47 for Writing to 0.84 for the Composite. 
Relative to Reading and Science, larger correlations were observed for English (0.79) and 
Mathematics (0.81). 

Table 7 


Longitudinal Correlations 


Grade Level 
Pair 

Subject Area 

English 

Mathematics 

Reading 

Science 

Writing 

Composite 

3-4 

0.60 

0.60 

0.72 

0.73 

0.37 


4-5 

0.66 

0.58 

0.72 

0.72 

0.47 


5-6 

0.68 

0.63 

0.71 

0.73 

0.32 


6-7 

0.67 

0.64 

0.70 

0.75 

0.47 


7-8 

0.70 

0.72 

0.69 

0.75 

0.47 


8-9 

0.81 

0.78 

0.74 

0.73 

0.64 

0.89 

9-10 

0.81 

0.81 

0.73 

0.74 

— 

0.89 

9-11 (ACT) 

0.81 

0.81 

0.73 

0.72 

— 

0.88 

10-11 (ACT) 

0.79 

0.81 

0.66 

0.65 

0.47 

0.84 


Gain score means and standard deviations (Table 8) allow comparisons of gain scores 
across grade levels and subject areas. For all subject areas except Writing, all of the mean gain 
scores are positive, suggesting that students typically increase their Aspire-tested knowledge and 
skills after one year of schooling. The means and standard deviations of the gain scores vary 
considerably by grade level. The largest mean gain score was observed for grade 7-8 Science 


(4.6), while the smallest was observed for grade 6-7 Reading and grade 8-9 Reading (1.5). 
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Recall that for Writing, gain scores do not have a clear meaning and one should not 
necessarily expect positive mean gain scores because of the nature of the Writing scales. The 
mean gain scores for Writing ranged from 5.9 for grades 5-6 to -3.3 for grades 6-7. Because of 
the smaller sample sizes for Writing, the mean gain scores are subject to greater sampling error. 
Writing scores increased substantially from grade 5 to grade 6 (mean gain score of 5.9), and then 
declined from grade 6 to grade 7 (mean gain score of -3.3) and grade 7 to grade 8 (mean gain 
score of -2.8). One possible explanation for the increase in scores for grade 6 is that the grade 6 
Writing test is a narrative writing exercise, unlike the grades 4 & 7 and 5 & 8 tests which are 
exercises in expository writing and persuasive argumentation, respectively. While raw scores of 
4 indicate “proficient” across all grade levels, the data suggest that students performed better on 
the grade 6 narrative writing exercise, perhaps because of greater comfort and familiarity with 
the narrative mode. 4 At grade 6, the ACT Aspire Writing score range expands from 408-440 to 
408-448, with a maximum raw score of 6 (instead of 5) for each domain score. It is likely that 
this scale increase is also partly responsible for the increase in scores from grade 5 to grade 6. 
Table 8 

Gain Score Means and Standard Deviations 


Grade 

Level 

Pair 

Subject Area 

English 

Mathematics 

Reading 

Science 

Writing 

Composite 

3-4 

4.3 (5.3) 

4.0 (3.7) 

3.4 (4.1) 

3.9 (4.6) 

-0.2 (6.0) 


4-5 

3.5 (5.5) 

3.2 (4.4) 

3.3 (4.4) 

3.1 (4.8) 

0.4 (6.5) 


5-6 

2.7 (6.0) 

3.4 (4.8) 

3.5 (4.9) 

2.6 (5.0) 

5.9 (8.1) 


6-7 

2.4 (6.8) 

1.6 (5.4) 

1.5 (5.0) 

2.3 (5.3) 

-3.3 (7.0) 


7-8 

2.8 (6.6) 

2.9 (5.4) 

4.3 (5.3) 

4.6 (5.4) 

-2.8 (6.2) 


8-9 

2.3 (5.9) 

1.7 (5.1) 

1.5 (5.1) 

1.7 (5.6) 

-1.0 (5.4) 

1.8 (3.4) 

9-10 

2.5 (5.9) 

2.7 (5.4) 

2.7 (5.3) 

2.0 (5.9) 


2.5 (3.4) 


4 Chrys Dougherty suggested that narrative writing requires storytelling, which students may find inherently easier 
because the sequence of events in a story help students organize the information (personal communication, October 
8 , 2014 ). 
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The proportion of examinees with positive one-year gain scores is provided in Table 9. It 
is not uncommon for student scores to decline after one year. For all subjects except Writing, 
among all gain scores observed for grade level pairs 3-4 through 7-8, 21.8% were negative and 
71.4% were positive. Similar to the gain score means, there is considerable variation across 
grade levels and subject areas. The highest proportion was observed for grade 3-4 Mathematics 
(0.83) and the lowest proportion was observed for grade 8-9 Reading and Science (0.54). It 
should be expected that grade level pairs and subject areas with larger mean gain scores would 
have a higher proportion of positive gain scores, and that the resulting predicted path estimates 
would include a greater share of positive gain scores. 

For Writing, the proportion of examinees with positive one-year gain scores varied from 
0.27 for grade level pairs 6-7 and 7-8 to 0.71 for grade level pair 5-6. Again, because of the 
nature of the Writing scale, one should not necessarily expect a large proportion of positive gain 
scores. 

Table 9 

Proportion with positive one-year gain scores 


Grade 

Level 

Pair 

Subject Area 

English 

Mathematics 

Reading 

Science 

Writing 

Composite 

3-4 

0.78 

0.83 

0.76 

0.77 

0.42 


4-5 

0.71 

0.72 

0.74 

0.70 

0.45 


5-6 

0.64 

0.73 

0.73 

0.66 

0.71 


6-7 

0.63 

0.57 

0.59 

0.64 

0.27 


7-8 

0.64 

0.67 

0.77 

0.78 

0.27 


8-9 

0.58 

0.55 

0.54 

0.54 

0.33 

0.66 

9-10 

0.61 

0.62 

0.66 

0.57 


0.74 
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Selected Predicted Path Graphical Results 

Next, we examine line graphs with the predicted path results. With 43 sets of predicted 
paths (8 grade levels by 5 subject areas, plus Composite paths for grades 8-10), we only present 
selected results graphically. Figure 3 shows the one-year predicted paths for grade 3 ACT 
Aspire Science scores. The grade 3 scores are presented on the horizontal axis and predicted 
grade 4 scores are presented on the vertical axis. The blue line represents the lower bound of the 
predicted path and the red line represents the upper bound. The dashed black line represents the 
predicted grade 4 score that is used in the calculation of predicted mean scores for aggregate 
reports. Note that the predicted grade 4 scores are rounded to the nearest tenth, while the lower 
and upper bounds are rounded to the nearest integer. Therefore, the dashed black line appears 
smoother. The solid black line, provided for reference, represents the case where grade 3 score 
is equal to grade 4 score (gain score=0). All lines are monotonically increasing. The width of 
the one-year predicted path ranges from 5 (e.g., 427-431 for a grade 3 score of 428) to 8 (e.g., 
409-416 for a grade 3 score of 407). The width of the predicted path decreases with higher grade 
3 scores. 

For grade 3 Science scores of 417 and lower, the paths predict strictly positive gain 
scores. This means that all values within the predicted path imply a positive gain score. For 
grade 3 Science scores of 424 and higher, the predicted paths include positive and negative gain 
scores. There are no grade 3 Science scores for which the paths predict strictly negative gain 
scores. Later, we examine the relative frequency with which the paths predict strictly positive 
and strictly negative gain scores. In Figure 3, the intersection of the grade 3 and grade 4 
Benchmarks is marked with the green asterisk. The one-year predicted path for the grade 3 
Benchmark includes the grade 4 Benchmark. 
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Figure 3 

One- Year Predicted Paths for Grade 3 Science Scores 


— Lower Bound — Predicted — Upper Bound Gain=0 X Benchmark 



Figure 4 shows the two-year predicted paths for grade 3 ACT Aspire Science scores. By 
definition, the two-year predicted score ranges are twice as wide as the one-year ranges. Figure 
4 also shows that a student who meets the grade 3 ACT Readiness Benchmark for Science (418) 
is predicted to score between 418 and 430 on the grade 5 ACT Aspire Science assessment. This 
range includes the grade 5 ACT Readiness Benchmark for Science (422). 
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Figure 4 

Two-Year Predicted Paths for Grade 3 Science Scores 


— Lower Bound — Upper Bound — Gain=0 x Benchmark 



Figure 5 shows the one-year predicted paths for grade 8 Reading scores. In this case, the 
predicted paths were estimated using the concordance-derived longitudinal sample. The width of 
the one-year predicted path ranges from 6 to 8, with the width generally increasing with grade 8 
Reading score. The sample standard deviation of grade 9 Reading scores was 7.2 (Table 6), so 
the width of the predicted path ranges from 0.83 to 1.11 standard deviations of grade 9 Reading 
scores. For grade 8 Reading scores of 407 and lower, the paths predict strictly positive gain 
scores. For grade 8 Reading scores of 413 and higher, the paths include negative gain scores. 
None of the grade 8 Reading paths predict strictly negative gain scores. 
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Figure 5 

One- Year Predicted Paths for Grade 8 Reading Scores 


— Lower Bound — Predicted — Upper Bound Gain=0 x Benchmark 



ACT Aspire Grade 8 Reading Score 


Figure 6 shows the two-year predicted paths for grade 8 Reading scores. Again, by 
definition, the two-year paths are twice as wide as the one-year paths. Students who score 
between 420 and 440 on the grade 8 Reading test will have a two-year predicted path that 
includes the grade 10 ACT Readiness Benchmark for Reading (428). 
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Figure 6 

Two-Year Predicted Paths for Grade 8 Reading Scores 


— Lower Bound — Upper Bound — Gain=0 X Benchmark 



ACT Aspire Grade 8 Reading Score 


Figure 7 shows the predicted grade 11 ACT Mathematics score ranges for grade 10 
Mathematics scores. The predicted paths were estimated using the longitudinal sample of 
students tested with ACT Aspire in spring 2013 and the ACT in spring 2014. The width of the 
predicted path increases as grade 10 Mathematics score increases. A student who meets the 
grade 10 ACT Readiness Benchmark for Mathematics (432) has a predicted ACT Mathematics 
score range of 20-24 with a predicted mean value of 22.4. As expected, the predicted score 
range for the grade 10 Benchmark includes the ACT College Readiness Benchmark for 


Mathematics (22). 
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Figure 7 

Predicted ACT Mathematics Score Ranges for Grade 10 Mathematics Scores 

— Lower Bound — Predicted — Upper Bound X Benchmark 



ACT Aspire Grade 1 0 Mathematics Score 


Coverage Properties of Predicted Paths 

Because the one- year predicted paths are estimated using the 25 th and 75 th percentiles of 
the conditional test score distributions, one would generally expect that 50% of students will 
score within the one-year predicted paths. However, using an inclusive definition of path 
coverage (that is, students who have a score equal to the 25 th or 75 th percentile are counted as 
within the interval), the expected percentage is greater than 50%. The one-year predicted paths 
should also have approximately symmetric coverage - that is, the percentages of students scoring 
above and below the predicted path should be similar. 

Table 10 reports the percentage of students scoring below, within, and above the one-year 
predicted paths. These percentages are based on the ACT Aspire-tested longitudinal samples. 
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For grade level pairs 3-4, 4-5, 5-6, 6-7, and 7-8, we see that the percentage of students scoring 
within the one-year predicted path ranges from 54% (grade 6-7 English) to 60% (grade 6-7 
Writing). For these grade level pairs, the predicted paths have mostly symmetric coverage. The 
largest level of asymmetry is observed for grade 6-7 Writing, where 22% of students scored 
below the predictive path and 18% scored above. 

For grade 10-11 (ACT Aspire to the ACT), coverage rates range from 56% (for English) to 73% 
for Writing. The coverage rate for Writing is high because the ACT Writing test has only 1 1 
possible score points, which leads to the 25 th and 75 th percentiles of the conditional score 
distribution encompassing a larger share of scores because of ties. The largest level of 
asymmetry is observed for Writing, where 16% of students scored below the predictive path and 
11% scored above. 

For grade level pairs 3-4, 4-5, 5-6, 6-7, 7-8, and 10-11, the same longitudinal samples 
used to estimate the predicted paths were used to evaluate the predicted paths. Thus, the level of 
coverage symmetry is likely higher than it would be if the estimation and evaluation samples 
were independent. 

For grade level pairs 8-9 and 9-10, the concordance-derived longitudinal sample was 
used to estimate the predicted paths and the ACT Aspire-tested longitudinal sample was used to 
evaluate the predicted paths. The evaluation samples, while small, are independent of the 
estimation samples. The one-year predicted path coverage rates ranged from 43% (grade 9-10 
Mathematics) to 87% (grade 9-10 Composite). There was less coverage symmetry than what 
was observed for the other grade level pairs. For example, for grade 8-9 Science, 64 students 
(41%) scored above their predicted path and 17 students (11%) scored below their predicted 
path. The large variability in coverage symmetry is expected from the small evaluation sample 
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sizes used for grades 8-9 and 9-10. In particular, for grades 9-10, the sample sizes ranged from 
only 15 (for Composite) to 26 (for English). Coverage asymmetry could also result from using 
the ACT Explore/ ACT Plan to Aspire concordance tables. 

Table 10 


Number and percentage scoring below, within, and above one-year predicted path 


Grade 

Level 

Pair 

Score 
Location 
Relative 
to Path 

Subject Area 

English 

Mathematics 

Reading 

Science 

Writing 

Composite 

3-4 

Below 

797 (21%) 

1,848 (21%) 

1,877 (21%) 

744 (22%) 

106 (23%) 


Within 

2,162 (57%) 

5,234 (59%) 

5,124 (57%) 

1,923 (57%) 

263 (56%) 


Above 

825 (22%) 

1,761 (20%) 

1,942 (22%) 

736 (22%) 

100 (21%) 


4-5 

Below 

854 (22%) 

1,825 (21%) 

1,886(22%) 

679 (22%) 

115(24%) 


Within 

2,211 (57%) 

4,987 (57%) 

4,915 (57%) 

1,734 (56%) 

268 (55%) 


Above 

847 (22%) 

1,912 (22%) 

1,893 (22%) 

657 (21%) 

104 (21%) 


5-6 

Below 

779 (22%) 

1,827 (23%) 

1,744 (22%) 

745 (22%) 

90 (21%) 


Within 

1,976 (55%) 

4,438 (56%) 

4,562 (57%) 

1,963 (57%) 

245 (57%) 


Above 

808 (23%) 

1,717 (22%) 

1,702 (21%) 

733 (21%) 

96 (22%) 


6-7 

Below 

527 (23%) 

1,286 (22%) 

1,320 (22%) 

369 (22%) 

82 (22%) 


Within 

1,261 (54%) 

3,293 (56%) 

3,297 (56%) 

948 (57%) 

229 (60%) 


Above 

541 (23%) 

1,297 (22%) 

1,263 (21%) 

360 (21%) 

70(18%) 


7-8 

Below 

428 (22%) 

1,009 (22%) 

1,023 (22%) 

473 (22%) 

55 (21%) 


Within 

1,070 (56%) 

2,537 (55%) 

2,573 (56%) 

1,209 (57%) 

154 (59%) 


Above 

426 (22%) 

1,037 (23%) 

983 (21%) 

450 (21%) 

53 (20%) 


8-9 

Below 

69 (29%) 

57 (22%) 

37 (31%) 

17 (11%) 

28 (24%) 

6 (18%) 

Within 

119(49%) 

142 (54%) 

54 (46%) 

75 (48%) 

67 (58%) 

19 (56%) 

Above 

53 (22%) 

62 (24%) 

27 (23%) 

64 (41%) 

21 (18%) 

9 (26%) 

9-10 

Below 

1 (4%) 

7 (33%) 

4 (20%) 

3 (14%) 


1 (7%) 

Within 

18 (69%) 

9 (43%) 

14 (70%) 

13 (62%) 


13 (87%) 

Above 

7 (27%) 

5 (24%) 

2 (10%) 

5 (24%) 


1 (7%) 

10-11 

(ACT) 

Below 

913 (23%) 

699(19%) 

747 (21%) 

742 (21%) 

144(16%) 

545 (19%) 

Within 

2,225 (56%) 

2,343 (62%) 

2,014 (57%) 

2,043 (58%) 

675 (73%) 

1,739 (61%) 

Above 

854 (21%) 

732(19%) 

779 (22%) 

740 (21%) 

103(11%) 

567 (20%) 


Next, we evaluate the coverage of predicted paths for EPAS data (Table 11). While not 
directly applicable to ACT Aspire, these findings will allow us to examine the coverage of the 
predicted paths using separate estimation and evaluation data sets. The EPAS data also let us 


examine the coverage of the two-year predicted paths. The percentage of scores falling within 
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the one-year predicted path ranged from 56% (grade 10-11 Reading) to 68% (grade 9-10 
Composite). The level of coverage symmetry varied across grade levels and subject areas. The 
largest levels of asymmetry were observed for grade 10-11 English (25% below, 16% above) and 
grade 8-9 Mathematics (22% below, 14% above). In these cases, it is likely that the 2014 cohort 
(the evaluation sample) demonstrated less growth than the 2013 cohort (the estimation sample), 
which would lead to a greater share of students scoring below their predicted path. 

For the two-year predicted paths, the coverage rates were typically much higher than 
what was observed for the one-year predicted paths. The exception to this was grade 9-11 
English, where the coverage rate was only 58%. The other two-year coverage rates ranged from 
71% (grade 9-11 Reading) to 88% (grade 9-11 Mathematics). The largest degree of coverage 
asymmetry was observed for grade 9-11 English, with 12% scoring below the two-year predicted 
path and 30% scoring above. Generally, students are more likely to score above, rather than 


below, the two-year predicted path. 
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Table 11 


Number and percent scoring within, above, and below one year and two-year predicted paths using EPAS 
data 


Grade 

Level 

Pair 

Score 
Location 
Relative 
to Path 

Subject Area 

English 

Mathematics 

Reading 

Science 

Composite 

8-9 

Below 

10,127(19%) 

11,586 (22%) 

10,630 (20%) 

10,799 (20%) 

9,885 (18%) 

Within 

34,312(64%) 

34,191 (64%) 

33,212(62%) 

34,312(64%) 

35,587 (66%) 

Above 

9,098 (17%) 

7,760(14%) 

9,695 (18%) 

8,426(16%) 

8,065 (15%) 

9-10 

Below 

30,074(17%) 

31,359 (18%) 

37,075 (22%) 

26,606(15%) 

23,743 (14%) 

Within 

104,377 (61%) 

115,077 (67%) 

99,114(58%) 

110,986(64%) 

117,287 (68%) 

Above 

37,888 (22%) 

25,903 (15%) 

36,150 (21%) 

34,747 (20%) 

31,309(18%) 

10-11 

Below 

36,591 (25%) 

27,055 (18%) 

33,683 (23%) 

35,192 (24%) 

31,604 (21%) 

Within 

88,059 (59%) 

89,711 (60%) 

82,357 (56%) 

88,655 (60%) 

92,368 (62%) 

Above 

23,706(16%) 

31,590 (21%) 

32,316(22%) 

24,509(17%) 

24,384(16%) 

8-10 

(2-year) 

Below 

29,231 (9%) 

32,176(10%) 

25,653 (8%) 

34,540(11%) 

27,935 (9%) 

Within 

274,305 (85%) 

248,898 (77%) 

245,789 (76%) 

253,167 (78%) 

259,021 (80%) 

Above 

20,205 (6%) 

42,667 (13%) 

52,299 (16%) 

36,034(11%) 

36,785 (11%) 

9-11 

(2-year) 

Below 

6,057(12%) 

2,486 (5%) 

5,171 (10%) 

5,120(10%) 

3,077 (6%) 

Within 

29,236 (58%) 

44,775 (88%) 

35,736 (71%) 

36,596 (72%) 

39,352 (78%) 

Above 

15,363 (30%) 

3,395 (7%) 

9,749(19%) 

8,940(18%) 

8,227(16%) 


Next, we examine how often the predicted paths include positive (or negative) gain 
scores. In Table 12, we present the percentage of Aspire-tested students of spring 2013 whose 
predicted path would be strictly positive (include only positive gain scores), strictly negative 
(include strictly negative gain scores), or other. 5 (For this analysis, we do not need to distinguish 
the one-year paths from the two-year paths because the two-year path is extrapolated from the 
one-year path, maintaining the same positive or negative trend). There is considerable variation 
in the percentages across grade levels and subject areas. Generally, the percentage of predicted 
paths that are strictly positive is larger for cases where positive gain scores are more likely. For 
example, the highest percentages for strictly positive correspond to grade 3 Mathematics (82%), 
grade 3 Science (74%), and grade 7 Science (71%), which also had among the highest 

5 Referring back to Figures 3, 5, and 7, strictly positive (negative) cases occur when the prediction interval lies 
entirely above (below) the gain=0 line. 
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proportions with positive gain scores (Table 9). Conversely, lowest percentages for strictly 
positive were observed for grade 6 English (1%), grade 6 Mathematics (1%), grade 8 English 
(1%), and grade 8 Science (3%), which also had the lowest proportions with positive gain scores. 
Low percentages of strictly positive predicted paths for grade 8 are due to the modest gains 
observed between grades 8 and 9. 

In some rare cases, strictly negative predicted paths are reported. This was most common 
for Writing, which is expected because of its scale construction and the differences across grade 
level pairs in mean gain scores. Among the other subject areas, strictly negative paths were most 
common for grade 6 Reading, with 2% of the paths predicting strictly negative gain scores. This 
is explained by the relatively small gain scores (mean=1.5, sd=5.0) observed for grade 6-7 
Reading (Table 8). 
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Table 12 


Number and percentage of students whose predicted path includes strictly positive or strictly negative 
gain 


Grade 

Level 

Gain 

Prediction 

Subject Area 

English 

Mathematics 

Reading 

Science 

Writing 

Composite 

3 

Positive 

11,155 (67%) 

14,631 (82%) 

11,422 (64%) 

12,218 (74%) 

1,403 (21%) 


Other 

5,511 (33%) 

3,160(18%) 

6,358 (36%) 

4,199 (26%) 

4,151 (62%) 


Negative 

0 (0%) 

0 (0%) 

0 (0%) 

0 (0%) 

1,155(17%) 


4 

Positive 

7,067 (41%) 

6,399 (36%) 

9,180 (54%) 

8,334 (51%) 

1,949 (32%) 


Other 

9,979 (59%) 

11,170 (64%) 

7,950 (46%) 

8,106 (49%) 

3,244 (53%) 


Negative 

0 (0%) 

0 (0%) 

0 (0%) 

10 (<0.5%) 

971 (16%) 


5 

Positive 

1,964(12%) 

8,361 (50%) 

7,698 (47%) 

3,430 (22%) 

2,571 (41%) 


Other 

14,219(88%) 

8,415 (50%) 

8,721 (53%) 

12,365 (78%) 

3,640 (58%) 


Negative 

0 (0%) 

0 (0%) 

0 (0%) 

0 (0%) 

96 (2%) 


6 

Positive 

79 (1%) 

159 (1%) 

3,621 (25%) 

1,107 (8%) 

503 (8%) 


Other 

14,388 (99%) 

14,802 (99%) 

10,362 (73%) 

12,960 (92%) 

3,306 (51%) 


Negative 

0 (0%) 

0 (0%) 

241 (2%) 

0 (0%) 

2,713 (42%) 


7 

Positive 

2,726 (21%) 

1,560(12%) 

8,196(64%) 

9,149 (71%) 

805 (12%) 


Other 

10,191 (79%) 

11,483 (88%) 

4,638 (36%) 

3,749 (29%) 

3,961 (60%) 


Negative 

24 (<0.5%) 

0 (0%) 

0 (0%) 

0 (0%) 

1,834 (28%) 


8 

Positive 

78 (1%) 

2,057(19%) 

430 (4%) 

334 (3%) 

1,615 (32%) 

0 (0%) 

Other 

10,579 (99%) 

9,034(81%) 

10,352 (96%) 

10,515(97%) 

3,173 (63%) 

12,061 

(100%) 

Negative 

0 (0%) 

5 (<0.5%) 

0 (0%) 

0 (0%) 

264 (5%) 

0 (0%) 

9 

Positive 

2,103 (26%) 

702 (9%) 

2,644 (35%) 

827 (11%) 

1,084 (35%) 

5,433 (56%) 

Other 

6,125 (74%) 

7,105 (91%) 

4,936 (65%) 

6,906 (89%) 

1,814(59%) 

4,204 (44%) 

Negative 

18 (<0.5%) 

0 (0%) 

18 (<0.5%) 

0 (0%) 

202 (7%) 

0 (0%) 


Summary 

Interpretations and limitations of the predicted paths 

The ACT Aspire predicted paths are designed to predict students’ next- year score ranges 
with greater than 50% accuracy and with approximate coverage symmetry (e.g., equal 
percentages of scores over- and under-predicted). The predicted paths cover two years, with the 
predicted score range for the second year formed by extending the one-year path in a linear 
fashion. Based on the EPAS data presented in Table 1 1, it is expected that the two-year 
coverage rates will be considerably higher than the one-year rates, with under-prediction more 


likely than over-prediction. 
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Given the design of the predicted paths, they support the following uses: 

• To determine if students are likely to meet ACT Readiness Benchmarks over the next two 
years 

• To determine which students are unlikely to meet a future-year standard and thus are 
candidates for extra academic support 

• To predict aggregate future achievement for a classroom, school, district, or state 

• To predict student achievement on the ACT (for grades 9 and 10) 

Students who score above their one-year predicted path will have demonstrated greater- 
than-expected growth; students who score below their one-year predicted path will have 
demonstrated less-than-expected growth. Generally, students who score above their one- year 
predicted path will have achieved a student growth percentile (Betebenner, 2008) greater than 
75; students who score below their one-year predicted path will have achieved a student growth 
percentile less than 25. 

The predicted paths for grades 3-7 and grade 10 were developed using students tested in 
spring 2013 and spring 2014. The predicted paths for grades 8 and 9 were developed using 
students tested 10-14 (or 22-26) months apart. Thus, interpretations of the one- year paths should 
assume a 12 month testing interval and interpretations of the two-year paths should assume a 24 
month testing interval. For students who test in the fall, the predicted paths assume that they will 
test again in the fall the following year. For example, if a student tests in fall grade 10 and plans 
to take the ACT in spring grade 11, their predicted ACT score range is likely a slight 
underestimate because there is more time remaining for growth than is assumed by the 


prediction. 



33 


Limitations of the predicted paths include: 

• Sample dependence. The predicted paths for grades 3-7 and grade 10 are developed 
using samples of students tested in spring 2013 and spring 2014, with a large percentage 
coming from one state (Alabama, the first state to administer ACT Aspire on a statewide 
basis). With larger samples of students, the estimation of the predicted paths will have 
less sampling error. It is also possible that predicted path estimates could shift upwards 
or downwards with the inclusion of more school districts and states in the sample. The 
predicted paths will be re-estimated as more data become available. 

• Concordance dependence. The predicted paths for grades 8 and 9 were developed using 
large samples of students tested with ACT’s EPAS system, with greater geographic 
diversity. However, this approach relies on the ACT Explore/ ACT Plan to ACT Aspire 
concordance, which could introduce bias into the estimation of the predicted paths. As 
more data become available, the predicted paths for grades 8 and 9 will be estimated 
using ACT Aspire data without the use of the concordance. 

• The two-year predicted score ranges, defined as a linear extension of the one-year ranges, 
will likely have asymmetric coverage. 

Future studies will monitor the predictive performance of the predicted paths and seek to 
identify steps that can be taken to improve the accuracy and interpretive value of the predicted 
paths. 



34 


References 

Betebenner, D.W. (2008). Toward a normative understanding of student growth. In K.E. Ryan 
& L.A. Shepard (Eds.), The future of test-based educational accountability (pp. 155-170). New 
York, NY: Taylor & Francis. 

Chen, C.L. (2005). An Introduction to Quantile Regression and the QUANTREG Procedure. 
SAS User’s Group International (SUGI) Paper 213-30. Downloaded July 3, 2014 from 
http://www2.sas.com/proceedings/sugi30/213-30.pdf . 

Koenker, R. (2005). Quantile Regression. New York, NY: Cambridge University Press. 
Koenker, R. & Bassett, G.W. (1978). Regression Quantiles. Econometrica, 46, 33-50. 


